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Abstract We consider the computation of stable approximations to the exact solution of 
, nonlinear ill -posed inverse problems F{x) =y with nonlinear operators F : X — > 7 between two 



Hilbert spaces X and Y by the Newton type methods 



^'{xfyF'ixf)) F'{xlr (f{xI) -y'-F\xl){xl -x,) 



in the case that only available data is a noise of y satisfying \\y^ —y\\ < 5 with a given small 
noise level 5 > 0. We terminate the iteration by the discrepancy principle in which the stopping 
index kg is determined as the first integer such that 



ih: ||F(xfj-/||<T5<||F(xf)-/||, 0<k<ks 



■s' 



00 

I with a given number T > 1. Under certain conditions on {a^}, {ga} and F, we prove that xf^ 



converges to x' as 5 ^ and establish various order optimal convergence rate results. It is 
remarkable that we even can show the order optimality under merely the Lipschitz condition on 
' the Frechet derivative F' of F if xq ~ x' is smooth enough. 
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1 Introduction 

In this paper we will consider the nonlinear inverse problems which can be formulated as the 
operator equations 

F{x)^y, (1.1) 

where F : D{F) C X ~^ Y is a nonlinear operator between the Hilbert spaces X and Y with 
domain D{F). We will assume that problem dl.lb is ill-posed in the sense that its solution does 
not depend continuously on the right hand side y, which is the characteristic property for most of 
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the inverse problems. Such problems arise naturally from the parameter identification in partial 
differential equations. 

Throughout this paper || • || and (•,•) denote respectively the norms and inner products for 
both the spaces X and Y since there is no confusion. The nonlinear operator F is always assumed 
to be Frechet differentiable, the Frechet derivative of F at x E D{F) is denoted as F'{x) and 
F'{x)* is used to denote the adjoint of F'{x). We assume that y is attainable, i.e. problem (1.1) 
has a solution E D{F) such that 

F(jc') =y. 

Since the right hand side is usually obtained by measurement, thus, instead of y itself, the avail- 
able data is an approximation y^ satisfying 

ll/-y|l<5 (1.2) 

with a given small noise level 5 > 0. Due to the ill-posedness, the computation of a stable solution 
of ( II. Il l from y^ becomes an important issue, and the regularization techniques have to be taken 
into account. 

Many regularization methods have been considered to solve (II. Il l in the last two decades. 
Tikhonov regularization is one of the well-known methods that has been studied extensively 
(see I.17.J1.19] and the references therein). Due to the straightforward implementation, iterative 
methods are also attractive for solving nonlinear inverse problems. In this paper we will consider 
some Newton type methods in which the iterated solutions {xf} are defined successively by 

4+1 =^o-ga, [F'ixfYF'ixf)) F'ixfr ^Fixf) -y'-F'{xf){xf -xo)) , (1.3) 

where Xq := xq is an initial guess of x^, {a^} is a given sequence of numbers such that 

«t>0, 1< — —<r and lim a*.. = (1.4) 

for some constant r > 1, and^a : [0,°°) (—00,00) is a family of piecewise continuous functions 
satisfying suitable structure conditions. The method il.3\ can be derived as follows. Suppose xf 
is a current iterate, then we may approximate F{x) by its linearization around xf, i.e. F{x) w 
F(xf) + F' (xf) (x — xf) . Thus, instead of (II. lb . we have the approximate equation 

F'{xf){x^xf)=y'~F{xf). (1.5) 

If F'(xf) has bounded inverse, the usual Newton method defines the next iterate by solving ( 11.5b 
for X. For nonlinear ill-posed inverse problems, however, F'{xf) in general is not invertible. 
Therefore, we must use linear regularization methods to solve (11.51) . There are several ways to 
do this step. One way is to rewrite ( 11.5b as 

F'ixf)h^y'-F{xf)+F'ixf)ixf -xq), (1.6) 

where h — x — xq. Applying the Unear regularization method defined by {ga} we may produce 
the regularized solution hf by 

hf = ga, [F'ixfYF'ixf)) F'ixfr {y'-Fixf)+F'{xf)ixf -xo)) . 

The next iterate is then defined to be xf^^ := xq + hf which is exactly the form ( 11.31 ). 

In order to use xf to approximate x^, we must choose the stopping index of iteration properly. 
Some Newton type methods that can be casted into the form ( 11.3b have been analyzed in ll3l[T2l 
[T4I under a priori stopping rules, which, however, depend on the knowledge of the smoothness 
of Xo — x^ that is difficult to check in practice. Thus a wrong guess of the smoothness will lead 
to a bad choice of the stopping index, and consequently to a bad approximation to x^. Therefore, 
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a posteriori rules, which use only quantities that arise during calculations, should be considered 
to choose the stopping index of iteration. One can consult ll3]l8ll4ll9ll2l fT4l for several such rules. 

One widely used a posteriori stopping rule in the literature of regularization theory for ill- 
posed problems is the discrepancy principle which, in the context of the Newton method dl.Sl l. 
defines the stopping index kg to be the first integer such that 

\\F{xl)~y'\\<T5<\\F{xf)~y'\l 0<k<ks, (1.7) 

where T > 1 is a given number. The method ( 11.31 ) with ga(A) = (a + A)^' together with ( 11.71 ) 
has been considered in |l3]l8l. Note that when ga{^) — (o: + A)^', the method ( 11.31 ) is equivalent 
to the iteratively regularized Gauss-Newton method iQ] 

xl, ^xl-{a,I + F\xlrF'{xl)Y' (F'{xlr{F{xl)-/) + a,{xl-x,)) . (1.8) 
When F satisfies the condition like 

F'{x)=R{x,z)F'{z)+Q{x,z), 
||/-/;(x,z)|| <C«||x-z||, ^,zeBp(/), (1.9) 

\\Q{xM<Cq\\F'{z){x-z)\1 

where Cr and Cq are two positive constants, for the method defined by dl.SI) and ( 11.7b with T 
being sufficiently large, it has been shown in |3 8| that if xq— satisfies the Holder source 
condition 

xo-x+ = (F'(x^)*F'(x^))^a) (1.10) 
for some o) G X and < V < 1/2, then 

|l4-xt||<o(52^/('+2v)). 

while if XQ — x^ satisfies the logarithmic source condition 

XQ-x'^ ^ {-\og{F'{x')*F\x^))y^ (0 (1.11) 
for some CO G X and > 0, then 

\\xl^-x^\<0{{-\n5)-n- 

Unfortunately, except the above results, there is no more result available in the literature on the 
general method defined by ( ll.3l l and ( ll.7l i. 

During the attempt of proving regularization property of the general method defined by ( ll.3l l 
and (11.7b . Kaltenbacher realized that the arguments in [T'Sl depend heavily on the special proper- 
ties of the function ga(A) = (a + A)^\ and thus the technique therein is not applicable. Instead 
of the discrepancy principle ( 11.71 ). she proposed in 1 13 1 a new a posteriori stopping rule to termi- 
nate the iteration as long as 

max{||F(xf,^_i)-/llJ^'(<-i)+^'(<-i)(<-<-i)-/ll}<T5 (1.12) 

is satisfied for the first time, where T > 1 is a given number Under the condition like ( 11.9b . it 
has been shown that if xq — x^ satisfies the Holder source condition ( 11.101 ) for some o G X and 
< V < 1/2, then there hold the order optimal convergence rates 

||xf,^-xt||<Cv||a)|r/(i+2v)52v/(i+2v) 

if {go} satisfies some suitable structure conditions, T is sufficiently large and \\(£)\\ is sufficiently 
small. Note that any result on ( 11.12b does not imply that the corresponding result holds for (11.7b . 
Note also that k^ < mg — 1 which means that ( |1.12b requires more iterations to be performed. 
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Moreover, the discrepancy principle (11.71 ) is simpler than the stopping rule (11.12b . Considering 
the fact that it is widely used in practice, it is important to give further investigations on (11.7b . 

In this paper, we will resume the study of the method defined by ( 11.3b and ( 11.7b with com- 
pletely different arguments. With the help of the ideas developed in Ii9i,19i,10l, we will show 
that, under certain conditions on {ga}, {oCk} and F, the method given by ( |1.3b and ( |1.7b indeed 
defines a regularization method for solving dl.lb and is order optimal for each 0<v<v — 1/2, 
where V > 1 denotes the qualification of the linear regularization method defined by {ga}- In 
particular, when xq — satisfies dl.lOb for l/2<v<V — 1/2, we will show that the order op- 
timality of (11.3b and ( 11.7b even holds under merely the Lipschitz condition on F' . This is the 
main contribution of the present paper We point out that our results are valid for any T > 1 . This 
less restrictive requirement on T is important in numerical computations since the absolute error 
could increase with respect to T. 

This paper is organized as follows. In Section 2 we will state various conditions on {go], 
{ttk] and F, and then present several convergence results on the methods defined by ( 11.3b and 
(11.7b . We then complete the proofs of these main results in Sections 3, 4, and 5. In Section 6, in 
order to indicate the applicability of our main results, we verify those conditions in Section 2 for 
several examples of {go] arising from Tikhonov regularization, the iterated Tikhonov regular- 
ization, the Landweber iteration, the Lardy's method, and the asymptotic regularization. 



2 Assumptions and main results 

In this section we will state the main results for the method defined by ( 11.3b and the discrepancy 
principle (11.7b . Since the definition of {x^^} involves F, ga and {o:a }, we need to impose various 
conditions on them. 

We start with the assumptions on ga which is always assumed to be continuous on [0, 1 /2] 
for each a > 0. We will set 

ra(A) :== 1 - A^a(l), 
which is called the residual function associated with ga- 
Assumption 1 Q faj There are positive constants cq and c\ such that 

0<ra(A)<l, ra(A)A<coa and < ga{^) < c\a^^ 

for alla>Q and A G [0, 1 /2]; 

(b) r„(A) <rp{X)foranyQ<a < P and X E [0,1/2]; 

(c) There exists a constant C2> such that 

rp{X)-ra(,X)<C2\l -rpi^) 

for any < a < j3 and A G [0, 1 /2]. 

The conditions (a) and (b) in Assumption [T| are standard in the analysis of linear regulariza- 
tion methods. AssumptionlTfa) clearly implies 

0<r„(A)A'/2 <c3ai/2 and < ga{X)X^I^ < c^a'^l^ (2.1) 

with C3 < c^J^ and C4 < c\^^ . We emphasize that direct estimates on ra(A)A^/^ and ga[X)X^I'^ 
could give smaller C3 and C4. From Assumption [Tf a) it also follows for each < V < 1 that 
ra(A)A^ < CqO^ for all a > and A G [0, 1 /!]. Thus the linear regularization method defined 
by {ga} has qualification V > 1, where, according to ll20l . the qualification is defined to be the 
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largest number v with the property that for each < V < V there is a positive constant such 
that 

ra{X)X'' <dva'' foralla >OandA e [0,1/2]. (2.2) 
Moreover, Assumption[TJa) implies for every ji > Q that 

raiX)i-lnX)-^ <min{(-lnA)-'',coaA-'(-lnA)-''} 

for all < a < Oo and A e [0,1/2]. It is clear that (-InA)"'' < (-ln(a/(2ao)))"'' for < 
A < a/(2ao). By using the fact that the function A cottA^' (— InA)^'^ is decreasing on the 
interval {Q,e^^] and is increasing on the interval [e^^, 1), it is easy to show that there is a posi- 
tive constant such that coaA"'(— InA)"'' < fl|j (— ln(a/(2ao)))"'' for a/(2ao) < A < 1/2. 
Therefore for every > there is a positive constant bf^ such that 

r„(A)(-lnA)-'' < (-ln(a/(2ao)))"'' (2.3) 

for all < a < OCo and A e [0, 1 /2]. This inequality will be used to derive the convergence rate 
when XQ — satisfies the logarithmic source condition dl.llb 

The condition (c) in Assumption [T] seems to appear here for the first time. It is interesting 
to note that one can verify it for many well-known linear regularization methods. Moreover, the 
conditions (b) and (c) have the following important consequence. 

Lemma 1 Under the conditions (b) and (c) in AssumptionU] there holds 

\\[r^{A*A)-ra{A*A)]x\\ < \\x- rpiA*A)x\\ + ^\\Ax\\ (2.4) 

V cc 

for all x,x £ X, any < a < j3 and any bounded linear operator A : X — s- F satisfying \\A\\ < 
1/V2. 

Proof For any < a < j3 we set 

Pp,aW--^ ''^^^^^'f^\ A e [0,1/2]. 
It follows from the conditions (a) and (b) in Assumption[T]that 

0<P/j,a(A)<min|l,c2-^|. (2.5) 

Therefore, for any x,x GX, 

\\[rp{A*A)-ra{A*A)]x\\ - \\pp,^{A*A)rp{A*A)x\\ 

<\\pp.aiA*A)[rpiA*A)x~x]\\ + \\pp^„iA*A)x\\ 
<\\rpiA*A)x~x\\ + \\pp^o^{A*A)x\\. (2.6) 

Let {El} be the spectral family generated by A* A. Then it follows from ( 12.5b that 

\\ppAA*A)x\\^ = fj^ [pp^a{>^)fd\\Eix\\^ 

= ^||Axl|2. 

a 

Combining this with ( I2.6I ) gives the desired assertion. □ 
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For the sequence of positive numbers {aic}, we will always assume that it satisfies il.4\ . 
Moreover, we need also the following condition on {cCk} interplaying with ra- 

Assumption 2 There is a constant c$> \ such that 

for all k and A G [0, 1 /2]. 

We remark that for some [go] Assumption |2] is an immediate consequence of ( |1.4| l. How- 
ever, this is not always the case; in some situations, Assumption|2]indeed imposes further condi- 
tions on {oLk}- As a rough interpretation, Assumption|2]requires for any two successive iterated 
solutions the errors do not decrease dramatically. This may be good for the stable numerical 
implementations of ill-posed problems although it may require more iterations to be performed. 
Note that Assumption|2]implies 

\\raM*A)A\<C5\\ra,^M*A)A\ (2-7) 

for any x£X and any bounded linear operator A :X satisfying ||A|| < 1 /\/2. 

Throughout this paper, we will always assume that the nonlinear operator F : D{F) cX 
is Frechet differentiable such that 

Bp {x"^) C D{F) for some p > (2.8) 

and 

||F'(x)|| <min{c3ao''^i3o''^}, xeBpix"^), (2.9) 

where < j3o < 1/2 is a number such that rag (A) > 3/4 for all X £ [0,/3o]. Since ?"a(,(0) = 1, 
such j3o always exists. The scaling condition ( 12.9b can always be fulfilled by rescaling the norm 
in Y. 

The convergence analysis on the method defined by (ll.3l l and il.li will be divided into two 
cases: 

(i) xq—x^ satisfies (II. 10b for some V > 1 /2; 

(ii) xo - Jc' satisfies (II. 10b with < V < 1/2 or ( II. lib with ^ > 0. 

Thus different structure conditions on F will be assumed in order to carry out the arguments. It 
is remarkable to see that for case (i) the following Lipschitz condition on is enough for our 
purpose. 

Assumption 3 There exists a constant L such that 

\\F'{x)~F'{z)\\<L\\x-z\\ (2.10) 

for all x,z£ Bp{x^). 

As the immediate consequence of Assumption[3] we have 

\\Fix)-Fiz)-F\z){x-z)\\<\L\\x~zf 

for all x,zE Bp(x^). We will use this consequence frequently in this paper. 

During the convergence analysis of ( 11.3b . we will meet some terms involving operators such 
as ra^{F'{xf)*F'{xf)). In order to make use of the source conditions ( II. 10b for xo—x\ we 
need to switch these operators with ra^. {F' {x^ )*F'{x'^)). Thus we need the following commutator 
estimates involving r^ and ga- 
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Assumption 4 There is a constant cg > such that 



II 



r„(A*A)-r„(B*B)|| <C6a 




(2.11) 



\\[ra{A*A)-ra{B*B)]B*\\<ce\\A-Bl 
\\A [ra{A*A) - ra{B*B)]B*\\ <C6a^l^\\A-B\ 



(2.12) 



(2.13) 



and 




(2.14) 



This assumption looks restrictive. However, it is interesting to note that for several important 
examples we indeed can verify it easily, see Section 6 for details. Moreover, in our applications, 
we only need Assumption]?] with A = F'{x) and B — F'{z) for x,z e Bp(x^), which is trivially 
satisfied when F is linear 

Now we are ready to state the first main result of this paper. 

Theorem 1 Let {ga} ond {ct^} satisfy Assumption\l\ ( 17.41 ), Assumption^ and Assumption^ 
let V > I be the qualification of the linear regularization method defined by {go}, <^nd let F 
satisfy ( 12. (St , ( 12.91 ) and Assumption\3\with p > 4||xo — Let {xf} be defined by ( 17.31 ) and let 
kg be the first integer satisfying ( 17.71 ) with T > 1. Let xq — satisfy U.lOi for some G) € X and 
1 /2 < V < V - 1 /2. Then 



if L\\u\\ < rjo, where u e [F' {x^ )*)-^ C Y is the unique element such that xq—x' = F'{x^)*u, 
T7o > is a constant depending only on r, T and Ci, and Cy is a positive constant depending only 
on r, X, V and Ci, / = 0, • • • ,6. 

Theorem[T]tells us that, under merely the Lipschitz condition on F', the method (|1.3l l together 
with (11.7b indeed defines an order optimal regularization method for each l/2<v<V— l/2;in 
case the regularization method defined by {ga} has infinite qualification the discrepancy princi- 
ple (11.7b provides order optimal convergence rates for the full range v G [1 /2, qq). This is one of 
the main contribution of the present paper 

We remark that under merely the Lipschitz condition on F' we are not able to prove the 
similar result as in Theorem[T]if xo~-x^ satisfies weaker source conditions, say ( II. 10b for some 
V < 1/2. Indeed this is still an open problem in the convergence analysis of regularization meth- 
ods for nonlinear ill-posed problems. In order to pursue the convergence analysis under weaker 
source conditions, we need stronger conditions on F than Assumption[3] The condition ( 11.91 ) has 
been used in 1 3 8 1 to establish the regularization property of the method defined by dl.SI ) and 
(11.7b . where the special properties of ga{^) = + oc)^' play the crucial roles. In order to study 
the general method ( 11.31 ) under weaker source conditions, we need the following two conditions 
on F. 

Assumption 5 There exists a positive constant Kq such that 




■^||<Cv||«||'/('+2^)52^/(l+2v) 



F'ix)=F'iz)R{x,z), 
\\I-R{x,z)\\ <7i:o||x-z|| 



for anyx,z & Bp{x'^). 



Assumption 6 There exist positive constants K\ and K2 such that 



\\[F'{x)-F'{z)]w\\<K,\\x-z\\\\F'{z)w\\+K2\\F'iz){x-z)\\\\w\\ 



for any x^Z € Bp{x^) and w G X. 
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Assumption |5] has been used widely in the literature of nonlinear ill-posed problems (see 
II17II1 ll l9l [T9l ); it can be verified for many important inverse problems. Another frequently used 
assumption on F is ( |1.9l l which is indeed quite restrictive. It is clear that Assumption|6]is a direct 
consequence of ( 11.91 1. In order to illustrate that Assumption |6] could be weaker than (11.9b . we 
consider the identification of the parameter c in the boundary value problem 

yu — g on aLl 

from the measurement of the state u, where i2 C M",n < 3, is a bounded domain with smooth 
boundary dO., f QL?{£2) andg e H^^^{dQ). We assume ^L?{Q) is the sought solution. This 
problem reduces to solving an equation of the form dl.lb if we define the nonlinear operator F to 
be the parameter-to-solution mapping F : L?{Q) I?{Q,),F [c) : = u{c) with u{c) £ H^{Q) C 
L^{Q) being the unique solution of ( 12.151 ). Such F is well-defined on 

D{F) := {ceL^{Q) : ||c-c||^2 < yfor some c > a.e.} 

for some positive constant 7 > 0. It is well-known that F has Frechet derivative 

F\c)h = -A{c)-\hF{c)), heL^in), (2.16) 

where A (c) : H^CiHq is defined by A (c)m —Au + cu which is an isomorphism uniformly 
in a ball Bp (c^ ) C D{F) around c' . Let V be the dual space of Ff' n Hq with respect to the bilinear 
form (9, ^) — Jq (p{x)\if{x)dx. Then A(c) extends to an isomorphism from L?{£2) to V. Since 
( 12.16b implies for any c,d G Bp{c^) and h £ L^{£2) 

{F'ic) - F\d)) h = -A(c)-' ((c - d)F'{d)h) - A(c)-i {h{F{c) - F{d))) , 

and since L} {Q) embeds into V due to the restriction n < 3, we have 

||(F'(c)-F'(«'))/.||^,<||A(c)->((c-«')F'(«')/z)||^, + ||A(c)-'(MF(c)-F(«f)))||^2 
<C\\{c-d)F'{d)h\\v^C\\h(F{c)-F{d))\\v 
<C\\{c-d)F'{d)h\\^, +C\\h{F{c)-F{d))\\i^i 
<C\\c-d\\ij\\F'{d)h\\^2+C\\F{c)-F{d)\\j2\\h\\i2. (2.17) 

On the other hand, note that F(c) — F(d) = — A(ii)^' ((c — d)F{c)), by using ( 12. 16b we obtain 

F{c) - F{d) - F\d){c - d) ^ -A{dy^ {{c - d) {F{c) -F{d))) . 
Thus, by a similar argument as above, 

\\F{c)~F{d)~F'{d){c^d)\y<C\\c-d\y\\F{c)-F{d)\\^2. 

Therefore, if p > is small enough, we have \\F{c) —F{d)\\i2 < C\\F'{d){c — d)\\i2, which 
together with ( 12.17b verifies Assumption|6] The validity of ( 11.9b . however, requires u{c) > K > 
forallceBp(c'), see Q. 

In our next main result. Assumption |5] and Assumption |6] will be used to derive estimates 
related to xf —x' and F'{x^){xf — x' ) respectively. Although Assumption|6]does not explore the 
full strength of ( 11. 9b . the plus of Assumption |5] could make our conditions stronger than ( 11. 9b 
in some situations. One advantage of the use of Assumption |5] and Assumption |6] however, is 
that we can carry out the analysis on the discrepancy principle ( |1.7b for any t > 1, in contrast to 
those results in ||3][8l where T is required to be sufficiently large. It is not yet clear if only one of 
the above two assumptions is enough for our purpose. From Assumption|6]it is easy to see that 

\\F{x)-F{z)-F'izKx-z)\\<\{K,+K2)\\x^z\\\\F'{z){x^z)\\ (2.18) 
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and 



\\F{x)-F{z)~F'{z)ix~z)\\<^{Ki+K2)\\x~z\\\\F'ix)ix-z)\\. (2.19) 



for any x,z e Bp{x'^). 

We still need to deal with some commutators involving ra- The structure information on F 
will be incorporated into such estimates. Thus, instead of Assumption]?! we need the following 
strengthened version. 

Assumption 7 (a) Under Assumption^ there exists a positive constant such that 

\\ra{F'{xyF'{x))-ra{F'{zyF'{z))\\<cjKo\\x-z\\ (2.20) 

forallx,z GBp{x^) and all a > 0. 

(b) Under Assumption\5\and Assumption^ there exists a positive constant c^ such that 

\\F'{x) [ra{F'{xrF'{x))-ra{F'{zrF'{z))]\\ 

<c^{KQ+K,)a'l^\\x-z\\+c^K2{\\F'{x){x^z)\\ + \\F'{z){x-z)\\) (2.21) 

for all x,z(z Bp{x^) and all a > 0. 

Now we are ready to state the second main result in this paper which in particular says that the 
method il.3\ together with the discrepancy principle defines an order optimal regularization 
method for each 0<V<V— 1/2 under stronger conditions on F. We will fix a constant 71 > 

C3ri/V(T-1). 

Theorem 2 Let {ga} and {cx^} satisfy Assumption\I] U-4i , Assumption^and Assumption^ let 
V > I be the qualification of the linear regularization method defined by {go}, ond let F satisfy 
i2.8\l . H2.9\l . Assumption\5\and Assumption^with p > 2(1 +C47i)||jco — -^^ll- Let {xf} be defined 
by il.3\l and let kg be the first integer satisfying H1.7\l with T > 1. Then there exists a constant 
Tji > depending only on r, X and cu i = 0, • • • ,8, such that if {Kq + Ki + K2)\\xq — < Tji 
then 

( i) IfxQ — jc^ satisfies the Holder source condition AlJOi for some (0 £X and < V < V — 1 /2, 
then 

<Cv||a)||i/(i+2^)52^/(^+2^), (2.22) 

where Cy is a constant depending only on r, X, V and Ci, / = 0, • • • ,8. 

(ii) IfxQ—x^ satisfies the logarithmic source condition U.lli for some CO G X and jJ. > 0, 
then 



\\4,~^'\\<c^\ 











1" n — n 


r 




ll«ll 





(2.23) 



where Cf^ is a constant depending only on r, X, pi, and Ci, i — 0, - ■ ■ ,8. 



In the statements of Theorem [T| and Theorem |2] the smallness of L||m|| and {Kq +Ki + 
K2)\\xq —x'^W are not specified. However, during the proof of TheoremlT] we indeed will spell 
out all the necessary smallness conditions on L||m||. For simplicity of presentation, we will not 
spell out the smallness conditions on {Kq +Ki +K2)\\xq —x''']] any more; the readers should be 
able to figure out such conditions without any difficulty. 

Note that, without any source condition on xq— x'', the above two theorems do not give 
the convergence of xf^ to x^ . The following theorem says that xf^ — > jc^ as 5^0 provided 

XQ— x'' e ^{F'{x'^))^ . In fact, it tells more, it says that the convergence rates can even be 
improved to o(52^/('+2^)) ifxQ-x'^ satisfies ( fTTOl ) for < V < V - 1/2. 
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Theorem 3 (i) Let all the conditions in Theorem\l\be fulfilled. Ifv>l and — xq satisfies the 
Holder source condition U-lOi for some CO G ^{F'{x^))^ one/ 1/2 < V < V - 1/2, then 

||4-.t||<,(52v/(U2v)) 

as 5 ^0. 

(ii) Let all the conditions in Theorem\2\be fulfilled. If xq — x^ satisfies ilJOi for some CO G 
andQ<V <V-\/2, then 

as 5^0. 

Theorem[T] Theorem |2] and Theorem[3]will be proved in Sections 3, 4 and 5 respectively. In 
the following we will give some remarks. 

Remark 1 A comprehensive overview on iterative regularization methods for nonlinear ill-posed 
problems may be found in the recent book [ 14„| . In particular, convergence and convergence rates 
for the general method (1.3) are obtained in lfT4l Theorem 4.16] in case of a priori stopping rules 
under suitable nonlinearity assumptions on F . 

Remark 2 In fW\ Tautenhahn introduced a general regularization scheme for dl.ll ) by defining 
the regularized solutions as a fixed point of the nonlinear equation 

x = XQ~ga {F'{x)*F'{x)) F'{x)* (f(x) -/ -F'(x)(x -xo)) , (2.24) 

where a > is the regularization parameter When a is determined by a Morozov's type dis- 
crepancy principle, it was shown in 1 18| that the method is order optimal for each < V < v/2 
under certain conditions on F . We point out that the technique developed in the present paper 
can be used to analyze such method; indeed we can even show that, under merely the Lipschitz 
condition on F' , the method in l,18j is order optimal for each l/2<v<V— 1/2, which improves 
the corresponding result. 

Remark 3 Alternative to (11.3b . one may consider the inexact Newton type methods 

4+1 - 4~8a, (F'(xf )*F'(xf )) F^xfr (Fixf) -/) (2.25) 

which can be derived by applying the regularization method defined by {ga} to ( 11.5b with the 
current iterate xf as an initial guess. Such methods have first been studied by Hanke in mm 
where the regularization properties of the Levenberg-Marquardt algorithm and the Newton-CG 
algorithm have been established without giving convergence rates when the sequence {ak} is 
chosen adaptively during computation and the discrepancy principle is used as a stopping rule. 
The general methods ( 12.251 ) have been considered later by Rieder in |15,161, where {ttjt} is 
determined by a somewhat different adaptive strategy; certain sub-optimal convergence rates 
have been derived when xq — x^ satisfies (II. 10b with Tj < v < 1/2 for some problem-dependent 
number < T] < 1 /2, while it is not yet clear if the convergence can be established under weaker 
source conditions. The convergence analysis of ( 12.251 ) is indeed far from complete. The technique 
in the present paper does not work for such methods. 

Throughout this paper we will use {xj^} to denote the iterated solutions defined by ( 11.31) 
corresponding to the noise free case. i.e. 

Xk+i=XQ-gai,{F'{xk)*F'{xk))F'{xk)* {F{xk)-y-F'{xk){xk-xo)) ■ (2. 26) 

We will also use the notations 

^:-F'(x+)*F'(xt), ^,:=F'(x,rF'(x,), ^/ F'(xf )*F'(xf ), 

^ F'(x+)F'(xt)*, F'{xk)F'{x,)\ := F'(xf )F'(xf )*, 
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and 

ek-=Xk-x\ ef-.^xf-x'^. 

For ease of exposition, we will use C to denote a generic constant depending only on r. T and 
c,-, / = 0, • • • , 8, we will also use the convention ^> < 'f' to mean that <P < for some generic 
constant C. Moreover, when we say -L||m|| (or {Kq + Ki + A'2)||eo||) is sufficiently small we will 
mean that -L||m|| < rj (or {Kq+Ki + A'2)||eo|| < rj) for some small positive constant Tj depending 
only on r, T and c,, / = 0, • • • , 8. 

3 Proof of Theorem [T] 

In this section we will give the proof of Theorem[T] The main idea behind the proof consists of 
the following steps: 

• Show the method defined by il.3i and ( |1.7| l is well-defined. 

• Establish the stability estimate \\xf — x^-H < S/^/aj^. This enables us to write \\efj\ < 

W^ksW + s/^/oj^. 

• Establish a^g > Cv(5/||co||)2/(i+2^) under the source condition ( fTTOl ) for 1/2 < v < v- 
1 /2. This is an easy step although it requires nontrivial arguments. 

• Show lleAg II < Cv II «|| which is the hard pait in the whole proof. In order 
to achieve this, we pick an integer ^5 such that A:g <ks ™d a^^ ~ (5/||(b||)^''''+^*''. Such^^ will 
be proved to exist. Then we connect He^.^ || and \\ei^ \\ by establishing the inequality 

\\ekJ<\\e-,J\ + ^{\\Fix,g)^y\\+d). (3.1) 

The right hand side can be easily estimated by the desired bound. 

• In order to establish ( 13. Il l, we need to establish the preliminary convergence rate estimate 

Ikfjl ^ IImII^/^S'/^ when xq - x^" =F'(x')*m for some u e JK{F'{x')*)^ cY. 

Therefore, in order to complete the proof of Theorem [T] we need to establish various esti- 
mates. 



3.1 A first result on convergence rates 

In this subsection we will derive the convergence rate ||e^^|| < IImII'/^S'/^ under the source 
condition 

xo-x"" ^F'ix^'yu, Me,yr(F'(/)*)-L. (3.2) 
To this end, we introduce kg to be the first integer such that 

5 

at<^-^<ak, 0<k<ks, (3.3) 
^ 7o||«ll 

where 70 is a number satisfying 70 > cor/(T — 1), and co is the constant from Assumption[T](a). 
Because of ( 11.41 ). such ks is well-defined. 

Theorem 4 Let {ga} and {cc^} satisfy Assumption\l\a), Assumption^ A2.12\l and ( 17.41 ), and let 
F satisfy ( 12.81 ), ( 12.91 ) and Assumption\3\with p > 4||jico — x^||. Let {xf} be defined by ( 17.51 ) and let 
kg be determined by the discrepancy principle ( 17.71 ) with T > 1. IfxQ — x^ satisfies ( 13.21 ) and if 
L||m|| is sufficiently small, then 
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(3.4) 



( i) For all < k < kg there hold 

xfeBpix^) and \\el\\<2{ci+cs,yQ)r^l^-al'^\\u\\. 

(ii) ks < kg' discrepancy principle ( 17.71 ) is well-defined. 
( Hi) There exists a generic constant C > such that 

ll<ll<C||«||'/25'/2. 

Proof We first prove (i). Note that p > 4||xo - jc^||, it follows from (IX2] | and ( |Z9] | that ( Il4b 
is trivial for k = 0. Now for any fixed integer Q < I <ks, we assume that i3.4i is true for all 
0<k <l.h follows from the definition (fT3] l of {xf} that 



4+1 



.ra,{s^!)e,~ga,{^u')F'{4r [F{xl)~y' ~F'{xl)ef) 
Using (13.21 ). Assumption[3] Assumptionlifa), ( 12.1b and ( 11.2b we obtain 

<lka.K')^"(^f)*«ll + lka.K')[^"(-^')*-^"(^f)1«ll 



(3.5) 
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+ c4a;'/'||F(xf)-/-F'(xf).f|| 
<c3a,^/'||«||+L||«||||.f|| + ic4L|kffa;'/'- 



Note that Sa^, ' < 7o||m|| for < ^ < ^5. Note also that at < ra^+i by (11.4b . Therefore, by using 
(13.4b with ^ = / — 1 , we obtain 



<2{c3+C4Yo)r^'^a 



(c3 + c47o)||m|| +L||m|| 



1/2„V2| 



1 



< 1. 



if L||m|| is so small that 

2(ri/2 + (c3+C47o)c4r)L|lM|l 

By using ( 13.51 ). ( 12.11 ). Assumption [51 ( 11.21 ). Assumption [TJ a), ( 13.4b with k = I - I and ( 13.61 ). we 
also obtain 



(3.6) 



< 



\\ra,_ii£^lll)eQ\\ +C4da, 



-1/2 , 1 



l|2„-l/2 



<||eo||+C47y'||«ir/'5i/2 + (c3 + C47o)c4r'/2L||M| 

<|ko||+c47y'll«ll'/'5'/2 + ip 

Therefore, by using p > 4||eo||, we have 

lkf||<Jp + c47o/'ll«ir/V/2<p 

if 5 > is small enough. Thus (13.4b is also true for all k ~ I. As I < kg has been arbitrary, we 
have completed the proof of (i). 

Next we prove (ii) by showing that kg <ks- From ( 13.51 ) and ( 13.2b we have for <k <ks that 

F'ix^)el, -y'+y = F'(xf )r„,(=e//) [^'(xf )* + )* - F'{xf y 



'F'{x^)-F'{xf)] rM') [F'i4r + {F'i^'r ~F'{xfr) 
F\x^)^F\xf)\ g„,(^/)^"(xf )* \Fixf) -yS-F\xf)ef 



-saM)'^t[F{4)-y-F'{4)4 

-ra,{^t){y'-y). 
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By using Assumption[3] Assumption [TJ a), (12.1b . ( 11.2b and ( 13.4b . and noting that 8/ak < 7o||m| 
we obtain 



||F'(/)ef+i-/+y|| <5 + coa,||«||+2c3L||«||a//2|kf||+L2||«||||ef||2 

+ c^L\\el\\5a-'l^ + 
<d + {cQ + ei)ak\\u\\, 



c4L|kf||5a;'/2 + ic4L2a;'/2||4lhV^L||.f||2 



where 



ei 



2ri/2 (c3 + C47o) (2c3 + C470) + 2(c3 + C47o)2, 



lIImII 



+ 4 



(c3+C47o)^r+(c3+C47o)^C4r^/2 L^\\uf , 



From (O, (l32]l and we have ||F'(;c''')eo -y^ +^11 < 8 + \\s^u\\ < 5+coaQ\\u\\. Thus, by 
using ( 11.41 ). 

\\F'{x^)ef^y^+y\\<5 + r{co + ei)ak\\ul 0<k<~ks. 

Consequently 

\\Fi4j~y'\\ < \\F'ix')el-y'+y\\ + \\Fixlj^y-F'{x^)el\\ 
<5 + r(co + ei)ajJ|M||+-L||efj|2 
<5 + r{cQ + ei +2{ci + C4YQ)^rL\\u\\)aiJ\u\\ 

<5 + r{co + ei+2{c3 + C4YofrL\\u\\)Yo^5 
< t5 



if L||m|| is so small that 

ei+2(c3+C47o)^rL|lM|| < 



2„M1..II ^ (T-l)70-Cor 



By the definition of kg, it follows that kg <ks. 

Finally we are in a position to derive the convergence rate in (iii). If kg = 0, then, by the 
definition of kg, we have ||F(jico) — y^H < t5. This together with Assumption[3]and (11.2b gives 

\\F'{x^)eo\\ < \\F{xo) -y-F'{x^)eo\\ + \\F{xo)~y\\ < ^-L\\eof + (t+ 1)5. 
Thus, by using ( 13.2b . we have 

lleoll - (eo,F'(xt)*«)l/2 = (F'(x+)eo,«)'/2 < |l/r'(/)eo|ll/2||„||l/2 



<yiL||«||||.o||+V7TT||«||'/25'/2. 



By assuming that L|1m|1 < 1, we obtain ||efj| = ||eo|| < ||m|| '/^S'/^. 

Therefore we will assume A;^ > in the following argument. It follows from (13.5b . ( 12.1b . 
Assumption|3]and ( 13.41) that for < ^ < ^5 

< ||r„,K^).oll+c45a;'^' + ^^4i|kfll'«r'^' 

< ||ra,K^)eo||+C4(T^||K||5)i/2 + (c3+C47o)c4r>/'i|l«lllkf|l. (3.7) 



14 



By ( 13.2b . ( I2.I2I 1 in AssumptionlH and Assumption[3]we have 

lka,K/)eo-r„,K)eo|| - II [r„,K«)-r„,K)]F' (/)*«! 

<C6||M||||F'(xf)-F'(x^)|| 



< C6L||m|| ||e^ 



(3.8) 



Thus 



<||r„,K>o||+C4(7o||«||5)>/2+(c6 + (c3 + C47o)c4r'/2)L||M||||4| 
<||r„,(=s/).o||+C4(7o||«||5)'/2^ 



ill 



if we assume further that 



4c5 (c6 + (c3 + C47o)c4ri/2^L||M|| < 1. 



(3.9) 



(3.10) 



Note that (|2.9l l and the choice of j3o imply ||rap(^)eo|| > 
induction we can conclude from ( 13. 9t that 



Thus, with the help of ( I2.7l i. by 



kfll <-C5||r„,KH||+C||«||'/25'/2, Q<k<h. 



This together with ( 13. 8t and ( |3.10t implies 

Ikfll <2c5||r„,K/)eo||+C||«||V25i/2^ 0<A:<fc5. 

The combination of ( I3.7l i. ( 13.1 lb and ( 13.101 1 gives 

lkf+il|<^||r„,K/)eo||+C||«||i/25'/2, 0<A:<^5. 
We need to estimate ||raj.(^/)eo||. By ( 13.2b . AssumptionlHa) and Assumption[3]we have 



(3.11) 



(3.12) 



|r„,K^).o|k = KK/).o,r„,«)r(xT)*« 



')F'{x') 

< ||F'(xfKK/)eo||||M|| +L||«||||ef||||r„,K/>o||. 



Thus 



lka,(<).o|| < ||F'(xf)r„,K^).oir/'||«||'/HL||«|||kfll- 
With the help of ( 13.51 ). ( 11.2b . AssumptionfTfa) and Assumption[3]we have 

||F'(xf)r„,K^).o|| < ll^"(-^f)^f+ill + lka.(^f)^f (F{xl)~y'^F'{xl)el) \\ 
<\\F{xl^,)~y'\\+25+\\F{xl,)~y~F'{xl,)el,\\ 

+ II [F\xl,)-F'{xl)]el,\\ + \\F{xl) ^y~F'{xl)el\\ 
<\\F{xl,)-y'\\+25 + L\\elf + 2L\\el,\\\ 

Therefore 



lka,k=4')^o|| < ||«||'/-|in-^f+i)-/||^/' + V2|k||i/25'/2 + v/2Z^ 



L||M|| + ViM Ikfl 
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k„,K/ko||<||«||^/'||^'(4+i)-/ll'/' + C||«||'/'5'/2 
1 



Combining this with (13.111 ) and ( 13.121 ) yields 

"\\F{4+i) 

(3V2 + 4C5) ViM + 4c5L||M||] ||r„,K/)eo|l 
(3V2 + 4C5) v^LlHI +4c5L||m|| < 1, 



Thus, if 



we then obtain 

lka,K>0|l<ll«|p/'llf'(^f+l)-/ll'/' + ll«ll'/'5>/2. 
This together with ( |3.12| i gives 

lkfll<ll«ir/^ll^'(4)-/ll'/^ + ll«ll'/^5'/2 

for all < A; < ^5 . Consequently, we may set k — k^in the above inequality and use the definition 
of/fc5toobtain||ef \\<\\ufl^5^l^. □ 



3.2 Stability estimates 

In this subsection we will consider the stability of the method ( 11.3b by deriving some useful 
estimates on \\x^ — where {xi^} is defined by ( 12.261 ). It is easy to see that 

ek+x = ra,{s^k)eG~ga,{^k)F'{xk)* {F{xk) ~y~F'{xt)ek) . (3.13) 

We will prove some important estimates on {xk] in Lemma[3]in the next subsection. In particular, 
we will show that, under the conditions in Theorem|4] 

Xk&Bp{x^) and \\ek\\ <2cir^l^aJ^\\u\\ (3.14) 

for all ^ > provided L||m|| is sufficiently small. 

Lemma 2 Let all the conditions in Theorem^and Assumption^ hold. If L\\u\\ is sufficiently 
small, then for all < k < kg there hold 

\\xt-Xk\\<2c4^ (3.15) 

V (^k 

and 

\\F{xf)-F{xk)-y^+y\\<{l+S2)5, (3.16) 

where 

£2 := 2c4 ((c6 + rc47o) + {4c3 + 3c^Yo)r^^^ +4{c3 + c^Yo)r'j L\\u\\ 
+ 4C3C4 f cer ' + (c4 + eg ) C3 r) 1 1 M I k . 



Proof For each < k < kg we set 

Uk:^F{x,)-y~F'{xk)e,, uf :^ Fixf) -y~ F'{xf)ef. (3.17) 
It then follows from ( l33b and ( 13.131 ) that 

4+i-^k+i=h+h + h+h, (3.18) 
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,K^)F'(xf)*(/-y), 



where 

By using (Il2] i. dTTTT i, dzni i. Assumption[3]and ( ITT4I ) we have 

Pill < ik„,K/)-r„,(^,)iiiiF'(xtr-F'(x,riiii«ii 

+ \\Va,{^k)-ra,{s^k)]F'{x,ru\\ 

") X —1/2 X 

<C6L ||M||||ei||||.5t^ -Xi.||a^ +C6L||m||||x^ -;ci|| 



<C6 (L||M||+2c3r^/2^2||j^|^ 
With the help of dZTT) and we have 



-■^^11 



l|/2||<C4 

By applying AssumptionlTla), (|2.141 l. Assumption |3] and ( |3.14t we can estimate It, as 

||/3|| < ka,(^.)r(xf)*-F'(x,)1«,|| + ||[g„,K,)-g„,K/)]F'(xf)*«,|| 
< (ci+C6)L||Mi||||xf-Xi,||a^"' < i(ci+C6)L2||e^||2||xf-Xi||a^"' 
<2(ci+C6)c^rL2||„||2||.^5_^^||. 



For the term I4, we have from (12.1b that 



P4|| < 



C4 



-M*:|| 



By using Assumption|3] ( 13.4b and ( 13.14b one can see 

\\u,-4\\<\\F{xl)~F{x,)^F'{x,){xl-x,)\\ + \\[F'{xl)-F'{x,)]el\\ 



< ^L\\x? 



-Xk\ 



-LWel 



114-^^11 < 2^ (3|kA'| 



\ek\ 



-Xk\ 



< (4c3 + 3c47o)r'/2a^i/2i||„||||x^_xi||. 



Therefore 



(3.19) 



II/4II < (4c3+3c47o)c4ri/2L||M||||xf-Xi||. 
Thus, if L||m|| is so small that 

(c6 + (4c3 + 3c47o)c4ri/2)L||M||+2(c3C6ri/2+c^(ci+C6)r)L2||„||2 < 1^ 
then the combination of the above estimates on Ii, I2, h and 74 gives for Q <k <kg that 



-■^it+lll < C4^= + -| 



This implies ( 13.15b immediately. 

Next we prove (13.16b . We have from (13.18b that 



F' (4 Kxt+i~xk+i)~y'+y = F' {x^ ) [h +h + h+ h) -y'+J- 



(3.20) 
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From (|321l, dTTlb . dTTjb . Assumption^ (ITT4] | and (ITTSl l it follows that 

||F'(xf)/i|| < ||F'(4)[r„,K^)-r„,(=24)][F'(xt)*-F'(x,)*]«|| 
+ \\F'{xl)[rM')-ra,{^k)]F'{x,ru\\ 
<C6L^||M||||ei||||xf-Xi.|| +C6L||M||a^'''^||xf -JCA;|| 
< [2c^ceL\\u\\ +'Xcj,c^C(,r^l^L^\\u\f^ 5. 

By using Assumptionllla) and ( 11.21 ) it is easy to see 

\\F'{xf )l2-y'+y\\ = \M^!){y'-y)\\ < d. 
In order to estimate F'{xf.)Ij,, we note that 

F'{xl)h = [f'{xI) -F'{xk)\ ga,{s^k)F'(xkru,+ [ra,{^!)-ra,m) 
Thus, it follows from (irTT l. Assumption]!] (IZTTT i. (ITT4l l and (ITTST l that 

||^"(xf)/3|| < II [F'(xf)-F'(x,)] 8a,i-^,)F'ix,ru,\\ 
+ 11 k "ill 



Mi-. 



< (C4 + C6)a^^ '''^Lllxf-.ti.llllMi.ll 
<^(C4 + C6)a;'/V|k,||2||xf 

< 4{c4 + C6)clc4rL^\\u\\^5. 



-XkW 



For the term F'{xf)l4 we have from Assumption[Tla), ( I3.19t and ( 13.151 1 that 

||^''(4)'f4|| < \\uk-uf\\<2{4c3 + 3c47o)c4r^^^L\\u\\5. 
Combining the above estimates, we therefore obtain 



\\F'{4m+i -xk+i) +.y|| < (1 + £3)5, < ^ < ^5, 



where 



(3.21) 



(3.22) 



£3 :=2c4 (^C6 + (4c3 + 3c47o)r'/2) L||m|| +4c3C4 (^cer^^^ + {c4 + ce)c3r^ i^^lH" 
This together with Assumption[3] (|3.4| l. ( 13.151 1 and jlAj impUes for < < ^5 that 



ll^'(^f+i)(^'+i-^i+i)-/+3'll 



<\\F\xf)ixl^-Xk+i)-y'+y\\+L\\xl,-xf\\\\xl,^x,+:\\ 



<(l+e3)5 + 2c4L(|kf+i| 



^fll) 



where 



Thus 



<(l+e4)5, 

£4 :=e3 + 8(c3+C47o)c4rL||M||. 
||F'(xf )(xf +3;|| < (1 + £4)5, 0<k<~ks. 
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Therefore, noting that d/ak<ry{)\\u\\ for < A: < ^5, we have 

||F(xf)-F(x,)-/+y|| < \\F{xl)-F{x,)-F\xl){xl-x,)\\ 

+ \\F'{xl){xl-x,)-y'+y\\ 

<iL||xf-x,||2 + (1+64)5 

<2c|L— 5 + (1 + 64)5 
< (l + e4 + 2rc|7oL||M||)5. 
The proof of (13.16b is thus complete. □ 



3.3 Some estimates on noise-free iterations 

Lemma 3 Let all the conditions in Theorem^be fulfilled. IfL\\u\\ is sufficiently small, then for 
all k>0 we have 

xteBpix^) and \\ek\\ <2c3r^^^al^^\\u\\. (3.23) 
If in addition, Assumption\l][b) is satisfied, then 



^\\ra,{^)eo\\ < H\\ < ^C5||r„,.(^>o|| (3.24) 



and 

z^lk^ll < Iki+ill <2|k,||. (3.25) 
2c5 

Proof By using ( I3.2l i. ( 12. 11 1. (I2.12l i and Assumption|3] we have from ( 13.131 1 that 

H+l-ra,{^)eo\\<\\K{s^k)~ra,{^)]F'(x^ru\\+^\\F{x^^ 



<C6L||M|||k^||+-^L||e^||2. (3.26) 



Since ( 12. Il l and i3.2\ imply ||rcn.(i/)eo|| < C3al^^\\u\\, we have 
Iki+ill < C3al^^\\u\\ +C6L||M||||ei|| +— 



1 

Note that (13.2b and ( 12.9b imply ||eo|| < C3aQ ||m||. By induction one can conclude the assertion 
( Il23b if Llkll is so small that 2(c6r^/2_|_p3p^^)^||(^|| < i 
If we assume further that 

5c5(c6 + C3C4r^''2)L||M|| < 1, (3.27) 
the combination of ( 13.261) and ( 13.231 ) gives 

\\ek+i-ra,i£/)eo\\ < {0^ + C3C4r^/^^ L\\u\\\\ei,\\ < ^J\ek\\. (3.28) 

Note that Assumption [Hb) and < 0:^^-1 imply ||rcu,(^)eo|| < ||rQ;j_, (j2/)eo||- Note also that 
AssumptionllJa) and (12. 9b imply (13.24b with k^Q. Thus, from ( 13.281 ) and (12.7b we can conclude 
(13.24b by an induction argument. ( 13.25b is an immediate consequence of (13.28b and ( 13.24b . □ 
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Lemma 4 Let all the conditions in Lemma^and Assumption\l\c} hold. If kg > and L\\u\\ is 
sufficiently small, then for all k>kg we have 

\\ek, II < |k,|| + {\\F{x,^ )-y\\+5). (3.29) 

V 

Proof It follows from (13.13b that 

+ ga,_,{.^k-i)F'ixk-iy [Fixk_i)-y-F'{x,_i)e,_i] . (3.30) 
Thus, by using ( |12] |, (irT2] i. Assumption|3] dZTT i. ( Il23] l and (|127] |, we have 

\\xks~Xk\\ < \\[ra,^_^{£/) - ra,_,{£/)]eo\\ +C6L||m|| (Ikn || + Iki^-i ||) 



< IIK,_,K) -'■a,_,K)HII + (Ik^-ill + ||%-i||) . (3.31) 

Since k>kg,we have ttk-i < Ctkg-i - Since Assumption[Tlb) and (c) hold, we may apply Lemma 
[T]with.;c = eo,x = e^^, a = ak-i, j3 = c^kg-i and A ~ F'{x'^) to obtain 

II i^) - ra,_, {^)]ea\\ < |ka,^_, (^)eo - e^J + ^=\\F' {x^)ekg \\ . 



Note that i3.2Si imphes 

1 

Note also that Assumption[3]implies 



11% K)^^o|| < 7— ll%-lll- 



||F'(xt).,^||<||FK)-y||+-L|k,,||^ 

Thus 

\\[ra,Aj^)-ra,_,i^)]eo\\ < -^|k,^-i || + (||F(x,J -y|| +L|k,J|2) , 
Since Lemma|2] Theorem]?] and the fact kg < kg imply 

\K\\<\\e!j+^<M^/'5^'\ 

we have 

IIK-i - K)]eo|| < II + ^ (II^^K) -yll +i-||«||5) . 

Combining this with ( 13.311 ) and using Lemma]3]gives 

\\xks ~xk\\ < Uck^W +C\\ei,\\+-^{\\F{xkg)~y\\+5) . 



This completes the proof. □ 
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3.4 Completion of proof of Theorem [T] 

Lemma 5 Assume that all the conditions in Lemma\3\are satisfied. Then 

\\F'{x')ek\\ < \\ra,i^)s^'^^eo\\+al%a,i^)eo\\ (3.32) 

/or all k>0. 

Proof We first use ( 13.131 1 to write 

F'{x^)ei,+i=F'{x^)ra,{£^)eQ+F'{x'^) [r„,Ki) - r„,(^)] eo 

-F'{x')ga,is^k)F'{xky [F{xk) -y-F'ixk)e,] . (3.33) 
Thus, it follows from ( Il2l ). Assumption^ AssumptionlTJa), jnH ), jnH ), ( Il23] l and (|324l i that 

\\F\x^)ek+i\\<\\F\x^ya,{s^)eo\\+L\\emra,{-^k)-ra,i^^^^^ 

+ ||F'(x,)[r„,(^,) - r„,(^)]F'(x' )*«|| + (1 +L\\ek\\a-''^)L\\ekf 

< ||r„,.KK'/'eo|| +L^MHf + (^'J^LMH\\ +L\\ek\\' 

<||r„,KK'/'eoll + ay'lk„,KHI|. 

This together with jZTT i and ( fL4] i implies ( |332] i. □ 

Lemma 6 Under the conditions in Lemma |2] and Lemma Q //'£2 < (t— l)/2 f/ie« /or f/ze 
determined by U.7\l with T > I we have 

(T- 1)5 < ||r„,K)=e/i/2eo|| +a,'/'||r„,K)eo|l (3.34) 

for allO<k< k^, 

Proof By using ( 13.16b . Lemma[3]and Lemma|5] we have for Q <k < kg that 

t5 < ||F(xf ) -/II < ||F(xf ) +y|| + ||F(x,) -y|| 

<(l+£2)5 + ||F'(xt)e,||+lL||e,||2 

< {l+e2)5 + C\\ra,(j^)^'/^eo\\+Cal^^\\ro,,{^)eo\\. 

Since t > 1, by the smallness condition £2 < (t — l)/2 on L||m|| we obtain (13.341) . □ 

Proof ofTheorem\l\ Ifkg = 0, then the definition of kg implies ||/^(xo) — y^|| < t5. From The- 
orem|4]we know that ||eo|| ^ ||m|| ^/^S'/^. Thus 

||^"(x^>o|| < ll^'(-^o) -3'-^"(-^')eo|| + ||F(xo) -/II +5 
<^-L\\eo\\^ + {l+x)5<8. 
Since eo — £/^(0 for some 1/2 < v < V — 1/2, we may use the interpolation inequality to obtain 

Ikfjl = Ikoll = \\^^(o\\ < ||a)||i/(i+2^)K^/2+^a)||2^/(^+2^) 

= ||a)||i/(i+2^)||F'(x^)eo||2^/('+2^) 

< ||a)||^/('+2*')52''/('+2''', 

which gives the desired estimate. 

Therefore, we may assume that ^5 > in the remaining argument. By using eg ~ £/^C0 for 
some l/2<v<V— 1/2 and Lemma |6] it follows that there exists a positive constant Cy such 
that 

{t - 1)5 <Cya^^^^^\\co\\, 0<k<ks. 



21 



Now we define the integer kg by 

.^_^.5,2/(l+2v) 



Then kg <kg. Thus, by using Lemma|2]and LemmalU we have 

Il4 II < \K II + ^ ^ 11% II + ' J- + ■ 



Note that Lemma|2]and the definition of kg imply 

II^K)-3'II<II^'(4)-/|| + II^'(4)-^K)-/+3'||<5. 
This together with ( I3.24l i. kg < kg and ||raj(^)eo|| < o:^||(o|| then gives 



< II < ay \\co\\ + ^ + -^< al 110)11 + (3.35) 



Using the definition of kg and (11.4b . we therefore complete the proof. □ 



4 Proof of Theorem |2] 

In this section we will give the proof of Theorem|2] The essential idea is similar as in the proof of 
Theorem[T] Thus we need to establish similar results as those used in Section 3. However, since 
we do not have source representation eo — F'[x^)*u any longer and since F satisfies different 
conditions, we must modify the arguments carefully. We will indicate the essential steps without 
spelling out all the necessary smallness conditions on {Kq + Ki +K2)\\e(i\\. We first introduce the 
integer ng by 

««,, < f — ) <«^' 0<^<n5. (4.1) 



J\\\eo 

Recall that 71 is a constant satisfying 71 > C3r'/^/(T— 1). 

Proof of Theorem^ In order to complete the proof of TheoremlH we need to establish various 
estimates. We will divide the arguments into several steps. 
Step 1. We will show that for all < < 

xfeBp(xt), |kf|l<|ko|l, (4.2) 

ll^"(-^^K'll<«;^'lko|| (4.3) 

and that kg < ng for the integer kg defined by the discrepancy principle ( 11.71 ) with T > 1 . 

To see this, we note that, for any Q<k<ng with G Bp (x^), ( 13.51 ) and Assumption|5]imply 

+ga,{^t)F'{4ny'-y)- 

Therefore, with the help of Assumption[lJa) and ( 12. It , we have 

lkf+ill<lko|| + ^^olkf|P + c45a;'/'<(l+c47i)lko|| + ^^olkfll'- 



Thus, if 2 ( 1 + C4 7i )/m;) I ko 1 1 < 1 , then, by using p > 2 ( 1 + C4 71 ) | ko 1 1 and an induction argument, 

4\ 



we can conclude |kf || < 2(1 +C47i)|ko|| < p forall < A: < ng. This establishes (|4r2] l. 
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Next we show ( 14.31) . It follows from ( 13.5b . Assumption [Ha), il.2i . ( |2.19t and ( 14.1b that for 

0<k<ns 

\\F'{xf)eU\<al%o\\+d+\\Fixf)-y^F'ixf)ef\\ 
<al%o\\ + {Ki+K2)\\ef\\\\F'{x')ef\\. 

By Assumption|6]we have 

\\[F'{x')-F'{xf)]el,\\<K,\\ef\\\\F'{x^^^^^^^ 
The above two inequalities and (14. 2b then imply 

ll^''(^^)4+lll<«.'^'lko||+ir,||.o||||^''(x^).f+ll| + (^l+^2)|ko|l||^^ 

Thus, if {Ki +^"2)116011 is sufficiently small, we can conclude (14.3b by an induction argument. As 
direct consequences of ( 14.2b . ( 14.3b and Assumption|6]we have 

\\F'ixf)ef\\<al%ol Q<k<ns (4.4) 

and 

ll^'(4+i)(4+i-4)ll<«i^'lko||, 0<^<«5. (4.5) 
In order to show kg < rig, we note that ( 13.5b gives 

F\x^)el, ~y'+y^F'(xf)ra,{^,')eo+[F'{x')^F'{xf)) r„,K')eo 

- (F'(xt)-F'(xf))^„,K/)F'(xf)* {Fixf)-y'-F'ixf)ef) 

~SaM)^l {f{4) -y-F'{xf )ef) ~ra,{:^l){y' ~y). 

Thus, by using iL2\ , AssumptionlUa), dOl l, Assumption|6] ( 12.181 ), ( |43] l, i4A\ and ( fTH ) we have 
for 0<k<ns 

\\F'{x^)el,-y'+y\\<8 + c,al'%o\\+c3KM\\4\^^^^^^ 

+ /ri||.f||(^5 + i(/r,+/r2)||.f||||F'(xf).f||) 

+ C4/r2a;'/'||F'(xf)4|| (^5+^-{K,+K2)\\ef\\\\F'ixf)ef\\^ 

+ ^-iK^+K2)\\ef\\\\F'ixf)ef\\ 

<8 + {ci+C{Ki+K2)\\eo\\)a/'-\\eo\\ 
<5 + r'^^{c3+C{Ki+K2)\\eo\ \ ) a^] \ \ eo \ \ . 

Recall that 71 > C3r'/^/(T- 1). Thus, with the help of ( 14.2b . ( 14.31 ) and the definition of ng, one 
can see that, if {Ki +K2)\\eo\\ is sufficiently small, then 

\\F{xf,^)~y'\\ < ||F(xfj-y-F'(x+)4|| + ||F'(/)4-/+y|| 

<d + r'^^{c3+C{Ki+K2)\\eo\ \ ) a„'f 1 1 eo 1 1 

+ I(7r,+7r2)|l4lll|f"(-^')4ll 

<5 + r'/^{c,+C{Ki+K2)\\eo\\)a!//\\eo\\ 
<8 + r'/^{ci + C{Ki+K2)\\eo\\)Y{'5 
< t5. 
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This implies kg <n§. 

Step 2. We will show, for the noise-free iterated solutions {jC/t}, that for all A: > 

lka,KHII<lk,||<|ka,K)eo||, (4.6) 

\\ek\\<\\ek+i\\<\\ek\\ (4.7) 

and for all < /t < Z 

|k,||<|k,||+^||F(x,)-3;||. (4.8) 
V 



In fact, from ( |3.13t and Assumption|5]it is easy to see that 

Iki+l - ra,{^k)eG\\ < ^^olk^lk- (4.9) 
If 2/ro||eo|| < 1, then by induction we can see that {x^} is well-defined and 

Iki:!! <2||eo|| foralU>0. (4.10) 
This together with ( |4.9l l and ( I2.20l i gives 

\\ek+i-ra,{j^)eo\\<\\[ra,i£^k)-ra,i.^)]eo\\+metf (4.11) 

Thus, by Assumption |2] and the smallness of AflljeoH we obtain ( 14.6b by induction. ( I4.7l i is an 
immediate consequence of ( 14.1 11 1 and (I4.6l l. 

In order to show ( 14.8b . we first consider the case ^ > 0. Note that x/^—xi has a similar expres- 
sion as in (13.30b . so we may use ( 12.20b . Assumption|5]and (14.10b to obtain 

\\xk-x,\\ < \\ra,_,{£/)eo-ra,_^{£/)eo\\+Ko\\eoU\\ek^i\\ + \\ei^i\\) 
+ Ko\\ek-if+KQ\\e,_i\\^ 
<IIK-,K)-'-a,-,K)ko||+^o|koll(lknll + lk/-ill)- (4.12) 
By Lemma[T]with X = eo, x = ek,a = ai^\, j3 = a^^i and A ~ F'{x'^), we have 

II i^) - ra,_, (^)]eo|| < |k%-, - ek\\ + -^\\F' {x^)ek\\. 

With the help of ( 12. 18b . ( 14. 10b . and the smallness of (/Ti +/r2)||eo||, we have 

\\F'{x')e,\\ < \\F{x,)-y\\ + ^-\\F'{x^)e,\\. (4.13) 

Therefore ||F'(jic^)e^.|| < 2\\F{xii) —y\\. This together with ( 14.1 lb and ( 14.7b then implies 

II ka,_, (^) - ra,_, K)]eo|| < ^o|ko|| Ik^ll + -^\\F{xk) -y\\ . 

V w 

Combining this with ( 14.121 ) gives 

\\xk-xi\\ </:olko|||kA-ll + Ik/ll + ^ll^(-«t)-3'll 

V 0!/ 



which implies (14.8b if ^TolkoH is sufficiently small. 

For the case = 0, we can assume / > 1. Since ( 14.8b is valid for = 1, we may use ( 14.7b to 
conclude that ( 14.8b is also true for k = 0. 

Step 3. We will show for all A: > that 

ll^"(^'KII < |ka,K)=<^'''%ll+«y'|ka,(-«^HI|. (4.14) 
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(4.15) 



(4.16) 
(4.17) 



To this end, first we may use the similar manner in deriving (14.3b to conclude 

ll^"(^^KII<«i^'lko||. 

Note that Assumption |6] and (14.1 Oi l imply 

\\[F'{x'')~F\xk)]ek\\ < iKi+K2)\\ek\\\\F'{x^)ek\\ 
<iKi+K2)\\eo\\\\F'ix')ek\\. 

Therefore 

|lF'(x,).,|l<|lF'(x+).,|l. 

In particular this implies 

\\F'{x,)e,\\<al%o\l 
By using (|3.33l l, ( |2.21| i. Assumption|6l ( 12.181 ) and Assumption[Tla) we obtain 

\\F'{x')e,+ i\\<\\ra,{^)j^'/^eo\\ + {Ko + Ki)\\ 

+ K2\\eo\\{\\F\x^)ek\\ + \\F\xk)ek\\) 

+ (/Ti +/:2)||e^||||F'(xi)ei|| +/i:2)lk^lPll/^'(xiH|| 

+ K2{K,+K2)\\e,\\\\F'{x,)e,fa;''\ 
Thus, with the help of (gUi, (|4TT5] |. (|4TT6] |, ( I4l7] i and ( I4l0] i, we obtain 

\\F\x^)ek+i\\<\\ra,{s^W''^eo\\+al%a,{^)eo\\+K2\\eoM^^ 

The estimates ( |4.14| i thus follows by Assumption |2] and an induction argument if /r2||eo|| is 
sufficiently small. 

Step 4. Now we will establish some stability estimates. We will show for all < < ng that 



ll^f-^^ll< — 



(4.18) 



and 



\\F{xf)~F{xk)-y^+y\\<{l+C{Ko + Ki+K2)\\eo\\)5. 



(4.19) 

In order to show ( |4.18t , we use again the decomposition ( |3.18t for xf^ ^ — 1 . We still have 
P2II < ^45/^/0^. By using ( |2.20t the term /i can be estimated as 

\\h\\<Ko\\eo\\\\xf~Xk\\. 
In order to estimate 73, we note that Assumption|5]implies 

^3 = [ga, [s^kWk - ga, {■s^k)-^k\ [Ri^'k -tek,Xk)- 1] Ckdt 
+ j\a,{^k )F'{xl)* \F'{xl)-F'{xk)\ [R{xk-tek,Xk)~I]ekdt 

r 5 1 

r-a,, {^k ) - ''a* {■^k) [R{h - tek,Xk) - 1] etdt 
+ j^^ gau{£^k)^k I~Rixk,4) [R{xk-tek,Xk)-I]ekdt. 
Thus, by using ( 12.201 ) and ( 14.101 ), we obtain 

l|/3ll<^o'lk^ll'lkf--^^ll<^olkoll'lkf--^^ll- 
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In order to estimate 74, we again use Assumption|5]to write 



ek 



^%K )< R{xl+t{x,-xl),xl)-I {x,-xt)dt 



I-RixkXk 

Hence, we may use ( 14.2b and ( 14.101 1 to derive that 

\\h\\<m4-Xkf+Ko\\ek\\\\4-Xk\\<Ko\\eo\\\\xf-Xk\\. 
Combining the above estimates we obtain for Q < k < ng 

ll-^f+i--*^-i+ill <-j=+KQ\\eo\\\\xf-Xk\\. 

V ^k 

Thus, if Afllleoll is sufficiently small, we can obtain (14. 18b immediately. 

Next we show (14. 19b by using (13.20b . We still have (13.21b . In order to estimate \\F' {xf)Ii\\, 
\\F'{xf)l3\\ and \\F'{xf)l4\\, we note that Assumption^ ( I4l0] i. ( l4l5T l and ( I4l8] l imply 

\\[F'ix,)-F'ix^)]{xf-x,)\\ 

<Ki\\ek\\\\F\x^)ixf-Xk)\\+K2\\F\x^)e,,\\\\xf-Xk\\ 
<Ki\\eo\\\\F'{x^){xf~Xk)\\+K2\\eo\\5, 



\\F\x,){xf~x,)\\<\\F'{x'){xf-x,)\\+5. 



which in turn gives 
Similarly, we have 

\\F\xf)ixf~x,)\\<\\F'{x^)ixf-x,)\\+5. 
Thus, by using dOTI ). ( I4l8] ). ( l430b and (g^B we have 



(4.20) 
(4.21) 



F'{xf)h\\ < iKo+Ki)\\eo\\a^'^\\xt~Xk\ 



1/2||„5 



+ ^:2|ko|| {\\F'{xf ){xf-x,]\\ + \\F\x,){xf-x,)\\ 



< 



{Ko+Ki+K2)\\eo\\5+K2\\eo\\\\F'{x^){xf~Xk)\ 



Moreover, by employing (3l2\ , (IZ20I 1. Assumption |6l (ITTSl i. (I4l0l l. (|437] |. (14081 1 and (E^Qll, 

||F'(x^)/3|| can be estimated as 

\\F'{xfM < (7^o+^i)||xf +a,-^''V2||F'(x,)(xf -^,)||||«,|| 
<iKo + Ki+K2)iKi+K2)\\eof5 
+ K2iKi+K2)\\eof\\F'ix^)ixf-Xk)\\. 

while, by using Assumption |6l dXTSI l. dOl i. (l4TT0l) . (14^ . (I4l8] l. (|430| | and (lOTl i. ||F'(xf )/4|| 
can be estimated as 

< (Ki +K2)\\xf -XkW \\F'ixk)ixf -Xk)\\ 

+ Kr\\xf-x,\\\\F'ixf)ef\\+K2\\F\xf){xf-x,meU 
<{Ki+K2)\\eo\\d+{Ki+K2)\\eo\\\\F'{x^){xf-x,)\\. 
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Combining the above estimates we get 

||F'(xf)(4+,-x,+i)-/+y|| 

< (i+c(/:o+/:i+/:2)lko||)5+c(/:i+/:2)||eollll^''(x')(xf-x^)||. (4.22) 

This in particular impUes 

\\F\xf)ixl,~x,+i)\\<5 + {K,+K2)\\eo\\\\F'{x^)ixf^x,)\\. 

On the other hand, similar to the derivation of (14.20b . by Assumption |6l ( 14.2b . (14.41 ) and ( 14.181) 
we have for < k < ns that 

||F'(xt)(xf+,-x,+,)ll<^2lkoll5+|lF'(xf)(xf+,-x,+,)ll- 

Therefore 

\\F'{x'){xf^i-Xk+i)\\<5 + iKi+K2)\\eo\\\\F'{x^)ixf-Xk)\\. 
Thus, if {Ki +K2)\\eo\\ is small enough, then we can conclude 

\\F'{x^)ixf~Xk)\\<5, 0<k<ns. (4.23) 

Combining this with (14.221) gives for < A: < ng 

\\F'{xf){xf^^~Xk+i)-y^+y\\<{l+C{Ko + Ki+K2)\\eo\\)5. (4.24) 

Hence, by using ( 14.24b . Assumption |6l (US, dOI ). ( 14.18b . ( 14.21b and (14.23b . we obtain for < 

k <n§ 

\\F'{xf){xf~Xk)-y^+y\\<{l+C{Ko + Ki+K2)\\eo\\)5. 

This together with dZTSl l. d^Il l and (l4TT0b implies ( |4TT9] l. 

Step 5. Now we are ready to complete the proof. By using the definition of kg, ( 14. 19b . (12.181) 
and (14.14b we have for < k < kg 

Td < WFixf) -/II < \\Fixf)-Fix,)-y' +y\\ + ||F(x,) ~y\\ 
< {l+C{KQ + Ki+K2)\\eQ\\)5 + C\\F'{x'^)ei,\\ 

<(l+C(/:o + ^:i+^2)|ko||)5 + C||r„,(^)^/'/2eo||+Cay^||r„,(^)eo||. 

Since t > 1, by assuming {Kq +Ki + A'2)||eo|| is small enough, we can conclude for Q <k < kg 
that 

{T-l)5<\\ra,{j^W^^eQ\\+al%a,{^)eo\\. (4.25) 

When xo — x'^ satisfies dl.lOb for some CO eX and < V < V — 1 /2, by using (14.25b . ( 14.8b . 
(14.6b . ( 14.18b . ( 14. 19b and the definition of kg, we can employ the similar argument as in the last 
part of the proof of Theorem[T]to conclude (12.22b . 

When XQ — jc^ satisfies dl.l lb for some (O £X and /i > 0, we have from Assumptionlifa) and 
(l23]lthat 

\\ra,{£^W'^^eo\\+al^\a,{-s^)eo\\ < (cq^/' + fe^) a,'/' (- ln(a,/(2ao)))"'' || «|| . 
This and (14. 25b imply that there exists a constant > such that 

(t- 1)5 < C^a^^ (-ln(a^/(2ao)))"'' ||a)||, < ^ < kg. 
If we introduce the integer kg by 

«r('l"K/(2«o)))"'' < ^-il^ ln(a,/(2ao)))-^ 0<^<^5, 
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then ks < ^5. Thus, by using d^Sl l. ( 14.181 ). ( |4.19t . the definition of ^5 and the fact < 
||''ai('2/)eo|| ^ (~ln(oii:/(2oco)))^''||a)||, we can use the similar manner in deriving (13.35b to 
get 

11411 ^ (-ln(%/(2«o)))"''|l«|l + ^ < -4=- (4.26) 



*5 



/OF 



By elementary argument we can show from jlAj and the definition of kg that there is a constant 



> such that 



0) 



In- 



to 



2m 



This together with ( |4.26t implies the estimate ( 12.231 1. 



□ 



5 Proof of Theorem |3] 

If XQ = x^, then kg =0 and the result is trivial. Therefore, we will assume xq 7^ x^. We define kg 
to be the first integer such that 

where the constant c > is chosen so that we may apply Lemma|6]or (|4.25t to conclude A:^ < kg. 
By (II.4I 1. such kg is clearly well-defined and is finite. Moreover, by a contradiction argument it 
is easy to show that 

kg ^00 as 5^0. (5.1) 

Now, under the conditions of Theorem |3] (i) we use Lemma |2] Lemma |4] and (|3.24t . while 
under the conditions of Theorem |3] (ii) we use (14.18b . (14. 19b . ( 14.6b and ( 14.8b . then from the 
definition of kg we have 



< 



< 



< 



1 



/a- 



fa-. 



■kg 



{\\F{x,^)^y\\+d) 



< 



ka, K)eo| 

o 



/CC: 



'ks 



< 



/OC; 



(5.2) 



'ks 



We therefore need to derive the lower bound of a^^ under the conditions on cq- We set for each 
a > and < < v 



Jo 



1/2 



where {E^ } denotes the spectral family generated by jz/. It is easy to see for each 0< jJ. <V that 
a"2M^„(A)2A2M is uniformly bounded for all a > and A e [0, 1 /2] and a-^f'railfl^^ 
as a for all A G (0, 1 /2]. Since (O G ■yy{F'{x^))^, by the dominated convergence theorem 
we have for each < ^ < v 

c^(a)^0 asa^O. (5.3) 
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By the definition of kg, ( 11.41 ). Assumption|2] and the condition cq = si/^co we have 



<||r„ K)^i/2.o||+a^J|r„ (^)eoll 



< 



This implies 



f c5 V''""' 
«t > ^ 7 7 ■ (5-4) 

Combining (15.2b and (15.41 ) gives 

/ \ l/(l+2v) , ,,, , , 

Since 0<V<V— 1/2, this together with ( 15. It and ( 15. 3t gives the desired conclusion. 
6 Applications 

In this section we will consider some specific methods defined by (ll.3l l by presenting several 
examples of {ga}- We will verify that those assumptions in Section 2 are satisfied for these 
examples. 

6. 1 Example 1 

We first consider the function ga given by 

A(a + A)'« ' ^^-'^ 

where m > 1 is a fixed integer. This function arises from the iterated Tikhonov regularization of 
order m for linear ill-posed problems. Note that when m= 1, the corresponding method defined 
by il3i is exactly the iteratively regularized Gauss-Newton method dl.St . It is clear that the 
residual function corresponding to (16.1b is 

a" 



(a + A)" 



By elementary calculations it is easy to see that Assumption[Tla) and (b) are satisfied with cq 
[m — l)™^'/m"' and c\ = m. Moreover ( 12. U is satisfied with 

1 /2ot-1\'" / fm+l 

C3 — —j^=^= — and Ci, — \ 1 — 



V2m - 1 V 2m / \ \m + 3^ 

By using the elementary inequality 

l-(l-f)"<\/^, 0<f<l (6.2) 
for any integer n > 0, we have for < a < j3 and A > that 



rf,i}-)-ra{}.) = r^iX) 



' 1+A/a ; 
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This verifies Assumption [He) with C2 = m^l^. It is well-known that the qualification for ga is 
V = m and ( 12.21) is satisfied with c/y = (v/m)^((m— v)/m)"'"^ < 1 for each < V < m. For the 
sequence {oLk] satisfying ( ll.4l l. Assumption|2]is satisfied with C5 = r"'. 
In order to verify Assumption]?] we note that 

ra{A*A)~ra{B*B) 

m 

= a!''Y.{aI+A*A)-\A*{B-A) + {B* -A*)B]{aI + B*B)-"'-^+\ (6.3) 
1=1 

Thus, by using the estimates 

\\{aI+A*A)-'{A*AY\\ < a-'+f" fori> 1 andO<Ai < 1, 

we can verify ( 12.1 11 1. ( I2.12l i and ( 12.131 1 easily. 

Note also that ga{^) = a"^ I™ 1 a'{a + X)-'. We have, by using faA2\ . 

m 

\\[ga{A*A)-ga{B*B)]B*\\ < a'' £ \\a'[{al +A*A)-' - {aI + B*B)-']B*\\ 

i=l 

<a-'\\A^B\\, 

which verifies ( 12.141 ). 

Finally we verify Assumption|2]by assuming that F satisfies Assumption|5]and Assumption 
|6] We will use the abbreviation := F'{x) for x e Bp (x^). With the help of dO) with A = F^ 
and B = F^, we obtain from Assumption|5]that 

^ra(F!^Fi)~ra(FtF^n 

m 

< a'"Y.\\{aI + Fi*Fi)-'Fi*Fi[R{z,x) ^ I]{al + f;*F^)-'"-^+'\\ 

i=l 
m 

+ a'''Y.\\{aI + F';F',)-\l^R{x,z)]*F^*F[{aI + F';FX'"'^^% 

;=1 

m m 

< a'" Y,a-'+^\\I - R{z,x)\\a-'"-^+' + a'" Y,a-'\\I ~ Rix,z)\\a-"'+' 

i=l i=l 

<\\I-R{z,x)\\ + \\I-R{x,z)\\ 
<Ko\\x~z\\ 

which verifies ( 12.201 ). In order to show ( 12.211 ). we note that, for any a £ X and b satisfying 
||fl|| = \\b\\ = 1, ( 16.31 ) implies 

(F:[r„(F:*F,')-r„(F-^;')]a,/.) 

m 

< a'" £ a-'+i \\{F^-F;){aI + F^*FX'"'^^'a\\ \\b\\ 
1=1 

m 

+ a"' £ a-"'-i/2+'|| (f: - + K*Ky'K*b\\ M- 
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Thus, by using Assumption |6] we have 

{F:,\ra{F:;F',)-ra{FtF^)]a,b) 
in 

<a'"Y,a''^^Ki\\x-z\\\\F^iaI + F^*Fl)-'"-^+-a\\ 

i=l 
m 

+ a'"Y^a-'+^K2\\Fi{x-z)\\\\{aI + F';F[)-"'-^+'a\\ 

i=\ 
m 

+ a'"Y,a-'"-^l^+'K4x- z\\\\F'^{aI + F';F'X'Fl*b\\ 

i=\ 
m 

+ a'" £ a-'"-i/2+%||f;'(x- z)|| \\{aI + F';F',)-'F';b\\ 

i=\ 

<Kia^l^\\x-z\\+K2{\\F',{x-z)\\ + \\F[{x-z)\\). 

This verifies ( 12.211 1. 

The above analysis shows that Theorem[T] Theorem |2] and Theorem[3]are applicable for the 
method defined by (11.31 ) and (11.71 ) with ga given by ( 16.11 ). Thus we obtain the following result. 

Corollary 1 Let F satisfy ( l2.iSI ) and ( 12.91 ), let {o:^} be a sequence of numbers satisfying ( 17.41 ), 
and let {xf} be defined by ( 17.31 ) with ga given by i6.1\l for some fixed integer m > 1. Let kg be 
the first integer satisfying ( 17.71 ) with T > 1. 

(i) If F satisfies Assumption\3\and if xq — x^ satisfies il.lOD for some (0 and 1/2 < V < 
m ~ 1 /2, then 

||xf^-X^||<Cv||«||'/(l+2v)52v/(l+2v) 

provided L||m|| < Tjo, where u £ J*/ (F' (x^Y")^ C Y is the unique element such that xq — jc' = 
F' (x^ )*u, rio>0 is a constant depending only on r, T and m, and Cy > is a constant depending 
only on r, X, m and V. 

(ii) Let F satisfy Assumption\5\and Assumption^ and let xq ~ Jc^ G (7^' (x^ Then there 
exists a constant rii > depending only on r, T and m such that if {Kq + Ki + K2)\\xo ~ x"' \\ <rji 
then 

limxl =x\ 

moreover, when xq —x^ satisfies ( 17.701 ) for some O) G X and 0<V<m— 1/2, then 

for some constant Cy > depending only on r, T, m and V; while when xq — x^ satisfies U.lli 
for some O) G X and jJ. >0, then 

\\xf^-x^\<C^\\(0\\(^l + 

for some constant Cfi depending only on r, T, m and pi. 

Corollary [T] with m = 1 reproduces those convergence results in mm for the iteratively reg- 
ularized Gauss-Newton method ( 11. 8t together with the discrepancy principle ( 11.71 ) under some- 
what different conditions on F. Note that those results in ||3]|8] require T be sufficiently large, 
while our result is valid for any T > 1 . This less restrictive requirement on T is important in nu- 
merical computations since the absolute error could increase with respect to T. Moreover, when 
xo — x^ satisfies ( II. 10b with v = 1/2, Corollary [T] with m = 1 improves the corresponding result 
in 131, since we only need the Lipschitz condition on F' here. 

Corollary [U shows that the method defined by (11.3b and ( 11.7b with ga given by (16.11 ) is or- 
der optimal for < V < m — 1/2. However, we can not expect better rate of convergence than 
Ql^g{2m-i)/ (2m) ^ q^q^ jf _ -^r satisfies ( 11.101 ) with m— 1/2 < V <m. An a posteriori stopping 
rule without such saturation has been studied in ll9l [T0l for the iteratively regularized Gauss- 
Newton method dl.Sb . 
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6.2 Example 2 

We consider the function ga given by 

[l/a] 

g„(A)= L (1-^)'' (6.4) 

!=0 

which arises from the Landweber iteration applying to Unear ill-posed problems. With such 
choice of ga, the method (11.3b becomes 

4+1=^0- E [l-F'{xfrF'{xf)) F'{xfr[F{xf)-y'-F'{xf){xf-xo)) 
which is equivalent to the form 

4,i+i ^4.i~F'{xfr [F{xf) -y'+F'{xf)(xi.-xf)) , < / < [l/a,], 

-^k+i — ^ir.fi/aj+i- 
This method has been considered in lfT2l and is called the Newton-Landweber iteration. 
Note that the corresponding residual function is 

r„(A) = (l-A)['/«l+^ (6.5) 

It is easy to see that AssumptionlHa), (b) and ( 12.11 ) hold with 

1 %/2 ^ 

cq = ^ ^' ^ ~ ^"'^ "^4 = v2. 

Moreover, by ( 16. 2t we have for any < a < /3 that 

rp(A) -r„(A) = rp(A) (l - (1 - A)['/«l-[l//^l) < ^^rplA). 

This verifies Assumption [TJc) with C2 = 1 . It is well-known that the qualification of linear 
Landweber iteration is v = oo and i2.2\ is satisfied with dy = for each < V < oo. 

In order to verify Assumption]!] we restrict the sequence {a/t} to be of the form a^t := l/n,, 
where {n^.} is a sequence of positive integers such that 

< HAr+i — Ha- < ^ and lim«i = oo (6.6) 
for some q> 1. Then for A G [0, 1/2] we have 

r„,(A) - (1 -A)«^-«^-+'r„,^,(A) < 2V..(^)- 

Thus Assumption|2]is also true. 

In order to verify Assumption|4] we will use some techniques from ir7lfT2ll and the following 
well-known estimates 

\\iI-A*AYiA*Ar\\<v'{j + v)-\ ;>0, v>0 (6.7) 

for any bounded linear operator A satisfying ||A|| < 1. 

For any a > 0, we set := [1 /a]. Let A and B be any two bounded linear operators satisfying 
||A|| , ||B|| < 1. Then it follows from dO that 

A- 

raiA*A)-raiB*B) ^ ^ (/ - A*A)-'' [A* (B - A) + (B* - A*)B] (/ - (6.8) 

j=0 
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By using ( 16.7b we have 

k 



J=0 



<Vk\\A-B\\ < -L\\A-B 



This verifies ( 12.1 It . 

From (HU) we also have A [ra (A* A) - Va {B*B)] B* =Ji+ J2, where 

k 

Ji := Y,{I~AA*yAA*{B^A){I-B*B)''-jB*, 

;=o 
k 

J2 ^A(/-A*A)^(B* -A*){I-BB*f-^BB*. 

j=o 

In order to verify ( 12.131 1. it suffices to show < 1)^'''^||A — B|| since the estimate on J2 
is exactly the same. We write Ji = y}'' +J^\ where 

■^1^' := L {I-AA*yAA*{B-A){I-B*Bf-jB*, 

yp);^ £ {I~AA*yAA*{B-A){I-B*Bf-JB*. 

j=[k/2] + l 

With the help of (16.71 ). we can estimate as 

ii-/p^ii< E (./•+ir'(fc+./-ir'/2||A-Bii 

;=[*:/2] + l 

< (^+ 1)-' 1 - < (^+ l)-'/2||A-B|l. 

In order to estimate we use AA* =7— (7 — AA*) to rewrite it as 

[k/2] 

yj') = £ (7-AA*)^'(B-A)(7-B*B)*-^B* 

;=o 

[V2] + l 

- £ (7-AA*)^(B-A)(7-B*B)'^+'-^B* 
j=i 

=(B - A) (7 - B*B)''B* - (7 - AA*) I'^/^l+i (B - A) (7 - 1*^/^]^* 

[V2] 

+ £ {I - AA*y {B ~ A){I - B*Bf-J {B*B)B* . 
;=i 

Thus, in view of ( 16.71 ). we obtain 

II <('t+ 1)-^''2||A -7?|| + {k- [k/2] + l)-i/2||A -7?|| 

[k/2] 

+ j:{k-j+i)-y'\\A-B\\ 

<{k+l)-^f^A-B\\. 
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We thus verify (12.13b . The verification of ( 12.12b can be done similarly. 
Applying the estimate ( 12.12b . we obtain 

\\[ga{A*A)~ga{B*B)]B*\\ <Y^\\ [iI-A*Ay-{I-B*By]B*\\ 

<k\\A-B\\<^\\A-B\\, 

which verifies ( 12.14b . 

Finally we verify Assumption |7]by assuming that F satisfies Assumption |5] and Assumption 
|6] From (16.8b and Assumption|5]it follows that 

raiFi*Fi)-raiF^*K:) ^t(^-Fi*FiyFi*Fi{R{z,x)-I)iI-F^*F^)'-^ 

k 

+ E {i-K*Kyii-R{x,z)yF!*F!{i^F^*F-^)''-j. 

7=0 

Thus we may use the argument in the verification of ( 12.13b to conclude 

\\r„iF;*F;)~ra{F!*F:)\\<\\I-Rix,z)\\ + \\I~R{z,x)\\<Ko\\x~z^^^ 

This verifies ( 12.20b . 

By using ( 16.81 ) and Assumption|5]we also have for any w EX 

Fi['-a{Fi*Fi) - ra{F:*F^)]w = Qx+Q2 + Qi + Qa, 

where 

21 = 1 {i-F'X*y{F:.F';yF:^-F',yi-F';F',f-^w, 

7=0 

22= t {I~FX*y{FX*){F[-F',){I-F':F'^'-^w, 

j=[kl2\ + \ 

[km 

23 - E {I~F^F^*yF^{I~R{x,z)yiF^*F^){I--F!*F!y-jw, 

.7=0 

G4= t iI-FiF!^*yFiiI-R{x,z)y{F!*F!yi-F^*F!y-Jw. 
By employing ( 16.71 ) it is easy to see that 

IIG3||< E(j + i)-'/'(fe-; + ir^||/-/?(x,z)||||w||<(fe+ir'/2/:o||x-z||||w||. 

7=0 

With the help of ( 16. 7b and Assumption|6] we have 

IIG2||< t ij+l)-'m-Fi)iI~FtF^y-^w^ 

j=[k/2\ + l 

<Ki\\x~z\\ t (; + ir'(^-; + i)-'/'llHI 

j=[k /2] + l 

+ K2\mx-z)\\ E 0-+l)-^||w|| 

j=\k/2] + l 

< (^+i)-i/2/:i||x-z||||w|l +^:2ll^;'(x-z)|l||w|l. 
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By using the argument in the verification of (12.13b and Assumption|6]we obtain 

IICill ^ II(^'-^,0(^-^'*^0*'vv|| + i|(f;-f;)(/-k'*^')^'"'''''^'vv|| 

[k/i] 

<(k+\)-'I^K,\\x-z\m\+K2\\F^{x-z)\\M\ 

+ l^[K,\\x-z\\{k-j + \)-'IHmF[{x-z)\\{k-j+l)-')\\w\\ 

.7=1 

<{k+\)-'I^K,\\x-z\\\M+K2\\Fl{x~z)\\\\w\\. 

Using Assumption|5]and the the similar argument in the verification of ( 12.131 1 we also have 

\\Q4<{k+lY'''^\I-R{^.z)\\M\<{k+l)-'I^K^\\x-z\\\\^v\\. 

Combining the above estimates we thus obtain for any w EX 

\\F'Ara{F::F:,)-ra{FtF:M 

<{Ko+K,)a'l^\\x-z\\\\w\\+K2\\F^{x-z)\\\\w\\ 

which implies ( |2.21t . 

Therefore, Theorem[Tl Theorem|2]and Theorem|3]are applicable for the method defined by 
( |1.3t and ( |1.7t with ga given by ( 16.41 ). 

The similar argument as above also appUes to the situation where ga is given by 

ga{X):^ 1(1 +Ar 

(=0 

which arise from the Lardy's method for solving linear ill-posed problems. 
In summary, we obtain the following result. 

Corollary 2 Let F satisfy ( 12. (SI ) and i2.9\l . and let {oCjt} be a sequence given by a^ — I /nj^, where 
{«it} is a sequence of positive integers satisfying ( |6.6| ) for some q>l. Let {x^} be defined by 
(O with 

[l/a] [1/a] 
^aW=E(l-Ay or E(l+A)-', 

1=0 !=0 

and let kg be the first integer satisfying ( 17.71 ) with T > 1. 

(i) If F satisfies Assumption\3\ and if xq — x^ satisfies ( 17.701 ) for some (0 X and V > 1/2, 
then 

||xf_^-/||<Cv||«||'/('+2^)52^/('+2v) 

provided L\\u\\ < rjo, where u e ^{F'{x^)*)-^ C Y is the unique element such that xq — x' = 
F'{x^)*u, 770 > is a constant depending only on T and q, and Cy is a constant depending only 
on X, q and V. 

(ii) Let F satisfy Assumption\5\and Assumption^ andlet xq — x'' € N{F' (x^))-^. Then there 
exists a constant T71 > depending only on x and q such that if [Kq +K\+ ^"2)11x0 — x'W < rji 
then 

]imxf =x\ 

moreover, when xq —x^ satisfies ( 17.701 ) for some (0 EX and V > 0, then 

\\xf -xt|l <Cv||t0||'/(l+2v)52v/(l+2v) 
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for some constant Cy > depending only on T, q and V; while when xq — satisfies M.ll^ for 
some CO €X and jJ. > 0, then 













) 




'"Nil 





for some constant Cfi depending only on T, q and jX. 



6.3 Example 3 

As the last example we consider the method (11.31 1 with ga given by 

g„(A) = |(l-.-^/«) (6.9) 

which arises from the asymptotic regularization for linear ill-posed problems. In this method, the 
iterated sequence {x^} is equivalently defined as x^j^^ := x^{\/ ttk), where x^ (f ) is the solution 
of the initial value problem 



j/{t) = F'ixlY {y'-F{xl)+F\xl){xl - x' {t))) , 
x^(0)-xo. 

Note that the corresponding residual function is 

It is easy to see that Assumptionflja), (b) and ( 12.11 ) hold with 

1 12 

Co ~ e , ci = 1, C3 = and C4 = 

\/2e 



f >0, 



By using the inequality 1—e ' < -/f for f > we have for < a < j3 that 

r/3(l)-r„(A) = r/3(A)(l-.^//^-^/«) < ^^-^r^(A) < /fr/3(A). 

This verifies Assumption [He) with C2 = 1. It is well-known that the qualification of the linear 
asymptotic regularization is v = 00 and ( 12.2b is satisfied with dv = (v/e)^ for each < V < 0°. 

In order to verify Assumption |2] we assume that {aic} is a sequence of positive numbers 
satisfying 

0<^^ ^ < 00 and lim «<, = (6.10) 

for some Oq > 0. Then for all A G [0, 1] we have 

r„,(A)=.('/«...-iM)^,„^^^(A)<,«o,^^^^(A). 

Thus Assumption|2]is also true. 

In order to verify Assumption|4]and Assumption]?! we set for every integer « > 1 

^a,n(A):= + — ) ^„.„(A):=i( + — 

V na J A \ V na 
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Note that, for each fixed a > 0, {ra.n} and {ga,n} are uniformly bounded over [0,1], and 
''a,n{^) fa{^) and §a,„(A) ga{^) as n °o. By the dominated convergence theorem, we 
have for any bounded linear operator A with ||A|| < 1 that 

hm||[r„(A*A)-r„.„(A*A)]x||2 

= hm / -ra,„ (A)) d{Ef^x,x)^0 

and 

lim||[g„(A*A)-g„,„(A*A)]x||2 

\\Af 



f" " 2 

lim / (ga(A)-ga,„(A)) d{Exx,x)^0 



for any x G X, where {E^} denotes the spectral family generated by A* A. Thus it suffices to 
verify Assumption |4] and Assumption |7] with ga and ra replaced by ga,n and ra^n with uniform 
constants cg, cj and cg independent of n. Let A and B be any two bounded linear operators 
satisfying ||A|| , ||B|| < 1. We need the following inequality which says for any integer n > 1 there 
holds 

\\ra.„{A*A){A*Ay\\<v''a\ < V < n. (6.11) 

By noting that 

ra,n{A*A)-ra,niB*B) 

^—f^raAA*A)[A*{B~A) + {B*-A*)B]ra,n+i-.iB*B), (6.12) 



we thus obtain 



\\ra,„{A*A)-rUB*B)\\<^^\\A~B\\, 

\\[ra,niA*A)-ra,n{B*B)]B*\\ < ^\\A-B\\ (6.13) 

and 

\\A[raAA*A)-ra,„iB*B)]B*\\<V2^\\A-B\\. 
Furthermore, by noting that ga.ni^) — ^ L"=i ''aji^), we may use (16.131 ) to conclude 

\\[ga,n{A*A)-ga.„{B*B)]B*\\ <—f^\\[ro,j{A*A)-raAB*B)]B*\\ 

<i-jA-Bi 

Assumption|4]is therefore verified. 

It remains to verify Assumption [T] with ga and Va replaced by ga.n and ra.n with uniform 
constants c-i and cg independent of n. By using ( |6.12| l. Assumption |5] and ( 16. lit we have 

\\ra,n{F';F'^-raAK*K)\\ 

\\raAFi*Fi){Fi*F^)iRiz,x)~I)ra.„+UF!*F!)\\ 

na .^j 

+ -^t\\'aAF:^*Fi)i^-Ri^^^)yiFt'F^,)ra.n^^^^^ 

na .^j 

< \\I-R{z,x)\\ + \\I-R{x,z)\\ 
<2Kq\\x-z\\. 
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This implies ( 12.201 ). 

By using ( 16.12b . Assumption |6] and (16.111) we also have for any a E X and b eY satisfying 

llflll = \\b\\ = 1 that 

<-^t\MFXliFXl{F:-Fi)ra,n+UFtF:)a,b)\ 

+ ^ t\i^,ra.n+i-,iFtF!)F!*{F!-F,l)F_:*raMFnb)\ 
< V2Kia^/'-\\x-z\\ +K2\\FAx-z)\\+^K2\\F_i{x^z)\\. 

This implies ( 12.211 ). 

Therefore, we may apply Theorem[Tl Theorem|2]and Theorem[3]to conclude the following 
result. 

Corollary 3 Let F satisfy i2.8\l and ( 12. 91 ), and let {oCjt} be a sequence of positive numbers satis- 
fying H6.10\l for some 9q > 0. Let {xf} be defined by ( I7.il ) with ga given by ( 16.91 ) and let kg be 
the first integer satisfying ( 17.71 ) with T > 1. 

(i) If F satisfies Assumption\3\ and if xq — x^ satisfies ( 17.701 ) for some (0 E X and V > 1/2, 
then 

|l4-X^||<Cv||«||'/(l+2v)52v/(l+2v) 

provided L\\u\\ < 770, where u € ^{F'{x^)*)-^ C Y is the unique element such that xq — x^ = 
F'(x^)*u, TJo > is a constant depending only on T, 0o cind Oo, and Cy is a constant depending 
only on T, do, Oq and v. 

(ii) Let F satisfy Assumption\5\and Assumption^ and let xq — x'^ € N{F' (x^))-^. Then there 
exists a constant r\i > depending only on x, and such that if {Ko-\-K\+K2)\\x() — x^\ < Tji 
then 

hmjcf =x'^; 

moreover, when xq —x^ satisfies UJOi for some (0 EX and V > 0, then 

||-<-xt||<Cv||a)||'/('+2^)52^/(i+2v) 

for some constant Cy > depending only on T, Oq, OCq and V; while when xq — x' satisfies liLlli 
for some CO EX and jl > 0, then 

\\xf^-x^\<C^\\(0\\(^l + 
for some constant Cfi depending only on T, 0o, CJCq and pi. 
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